Routine Outcome Monitoring and Clinical Feedback in Psychotherapy: Recent Advances and Future Directions.

In the past decade, there has been an increase in research related to the routine collection and active use of standardized patient data in psychotherapy. Research has increasingly focused on personalization of care to patients, clinical skills and interventions that modulate treatment outcomes, and implementation strategies, all of which appear to enhance the beneficial effects of ROM and feedback. In this article, we summarize trends and recent advances in the research on this topic and identify several essential directions for the field in the short to medium term. We anticipate a broadening of research from the focus on average effects to greater specificity around what kinds of feedback, provided at what time, to which individuals, in what settings, are most beneficial. We also propose that the field needs to focus on issues of health equity, ensuring that ROM can be a vehicle for increased wellbeing for those who need it most. The complexity of mental healthcare systems means that there may be multiple viable measurement solutions with varying costs and benefits to diverse stakeholders in different treatment contexts, and research is needed to identify the most influential components in each of these contexts.

Therapeutic techniques and session impact: A practice-research network study in private practice.

Objective: This study investigated the relationship between therapeutic techniques and session impact, by examining the replicability of findings observed in a university-based training clinic (Boswell et al., 2010) in another practice-oriented setting: private practice.Method: N = 8 therapists completed session-level assessments of their technique use for N = 38 clients. The same client sample completed session-level assessments of session outcome. Technique-outcome associations were examined with multilevel models.Results: As in Boswell et al., common factors were associated with positive session impact. For clients who received higher average common factor techniques (relative to their own therapist's caseload), session impact was the poorest in sessions with higher behavioral change techniques use (relative to the client's own average). Moreover, clients with the lowest average common factor techniques (relative to their therapist's caseload) reported better session impact in sessions that involved a higher degree of session-level behavioral change techniques (relative to their own average).Conclusion: In line with Boswell et al., therapists should be mindful of the consistency of their routine technique use between- and within-clients, and this can be aided through collection of their own practice-oriented data.

Bringing mental health to the frontlines: A proactive team-based model for healthcare workers during the COVID-19 pandemic.

OBJECTIVE: A novel team-based service was developed at the beginning of the pandemic in which sixty liaisons were assigned to provide proactive, tailored psychological support for healthcare workers (HCWs) across three of NewYork-Presbyterian's Weill Cornell affiliated hospitals. METHOD: The program took the proactive approach of bringing mental health awareness to every department and major division that interfaced with COVID-19 patients. Virtual and in-person team-based "town hall" meetings were offered to provide psychoeducation, facilitate discussion, foster adaptive coping and social cohesion, and identify employees who would benefit from further individualized support. RESULTS: The program's success was reflected in the number of town halls (1000+) and attendees (6000+) and in qualitative feedback from departments who requested ongoing services. CONCLUSIONS: This article presents the development, implementation, challenges, and opportunities in designing a team-based support model for HCWs. This model may be useful for organizations that seek to develop similar programs.

A proof-of-concept randomized crossover clinical trial of a first-in-class vasopressin 1a receptor antagonist for PTSD: Design, methods, and recruitment.

Background: Almost eight million Americans suffer from Posttraumatic Stress Disorder (PTSD). Current PTSD drug therapies rely on repurposed antidepressants and anxiolytics, which produce undesirable side effects and have recognized compliance issues. Vasopressin represents a promising and novel target for pharmacological intervention. Logistical issues implementing a clinical trial for a novel PTSD pharmaceutical are relatively uncharted territory as trials concerning a new agent have not been published in the past several decades. All published trials have repurposed FDA-approved psychoactive medications with known risk profiles. Our recruitment challenges are discussed in this context. Methods: An 18-week proof-of-concept randomized crossover clinical trial of a first-in-class vasopressin 1a receptor antagonist (SRX246) for PTSD was conducted. All participants received SRX246 for 8 weeks, the placebo for 8 weeks, and the drug vs. placebo arms were compared. Participants were assessed every 2 weeks for PTSD symptoms as well as other medication effects. Results were expected to provide an initial demonstration of safety and tolerability in this clinical population and potentially clinical efficacy in SRX246-treated patients measured by Clinician Administered PTSD Scale (CAPS) score changes, clinical impression, and other indices compared to placebo. The primary hypothesis was that SRX246 would result in a clinically meaningful 10-point reduction in mean CAPS score compared to placebo. Discussion: This study is the first to investigate an oral vasopressin 1a receptor antagonist for PTSD. As a wave of PTSD clinical trials with new pharmaceutical compounds are beginning now, lessons learned from our recruitment challenges may be invaluable to these endeavors.

How therapists and patients need to develop a clinical feedback system after 18 months of use in a practice-research network: a qualitative study.

BACKGROUND: A personalized computer-adaptive system for clinical feedback and routine outcome monitoring in mental health, Norse Feedback aims to bridge the needs for standardized and idiographic measures in ordinary practice. METHODS: Item response theory analyses of completed treatment processes (n = 800) informed a qualitative study comprising individual in-depth interviews and focus groups with patients (n = 9) and clinicians (n = 10). The research question was: How do clinicians and patients contribute to developing a clinical feedback system in a continuous process aimed at refining its clinical usefulness? RESULTS: We conducted thematic analyses and found five themes: 1. Added clinical needs, 2. Needs for re-organizing the clinician report, 3.Needs for differentiation of clinical content, 4. User-interface needs, and 5. Item level suggestions. CONCLUSION: In this article, we detail resulting needs for continuous adaptation to practice, and discuss implications of the concrete experiences with the Norse action research program for the larger field of ROM/CFS implementation.

Initial quantitative development of the Norse Feedback system: a novel clinical feedback system for routine mental healthcare.

PURPOSE: As routine outcome monitoring has become prevalent in psychological practice, there is need for measurement tools covering diverse symptoms, treatment processes, patient strengths, and risks. Here we describe the development and initial tests of the psychometric properties of a multi-scale system for use in mental healthcare, Norse Feedback. METHODS: In Study 1, we present the item-generation process and structure of the Norse Feedback, a 17-scale digital-first measurement tool for psychopathology and treatment-relevant variables. In Study 2, we present analyses of this initial measure in a nonclinical sample of 794 healthy controls and a sample of 222 mental health patients. In Study 3, we present the analysis of a revised 20-scale system in two separate samples of patients. In each analysis, we investigate item and test information in particular, including analysis of differential item functioning on gender, age, site, and sample differences where applicable. RESULTS: Scales performed variably. Changes to items and scales are described. Several scales appeared to reliably discriminate individuals entering mental health treatment on severity, and others are less reliable. Marked improvements in scale internal consistency and measurement precision were observed between the first and second implemented versions. CONCLUSION: This system includes some scales with reasonable structural validity, though several areas for future development are identified. The system was developed to be iteratively re-evaluated, to strengthen the validity of its scales over time. There are currently a number of limitations on inferences from these scores, which future developments should address.

Continuous quality improvement in measure development: Lessons from building a novel clinical feedback system.

PURPOSE: While the use of clinical feedback systems has become commonplace in psychological treatment, many of the most common instruments used for this purpose have not changed in decades. This paper describes the first four cycles of a measure development method designed to embrace continuous quality improvement. METHODS: Using techniques and philosophies developed in business management and academia-lean continuous quality improvement, action research, and practice research networks-we iterated through multiple cycles of development with the goal of creating an optimal clinical feedback system. These cycles emphasize building capacity to receive and implement feedback from a variety of stakeholders, especially patients and providers of behavioral health services, while also being responsive to quantitative findings from measure development. RESULTS: Iterating measure development with stakeholder feedback over the course of 5 years has resulted in a novel measurement system with 19 subscales administered via branching logic, and a supporting practice research network to sustain development. CONCLUSION: In developing a new clinical feedback system, the less-frequently-discussed practical aspects of measure development require close attention. Specifically, being willing to embrace change, planning for iteration, and systematically seeking stakeholder feedback are identified as central methods for improving clinical feedback systems.

Understanding routine outcome monitoring and clinical feedback in context: Introduction to the special section.

The practice of routine outcome monitoring and providing clinical feedback has been widely studied within psychotherapy. Nevertheless, there are many outstanding questions regarding this practice. Is it an evidence-based adjunct to ongoing psychotherapies, or an ineffective complication of treatment? If it is effective, through what mechanism(s) does it act? Is it effective with all patient populations, treatment types, and service delivery mechanisms, or does its impact vary across context? What choices in the implementation process affect the utility of patient-reported data feedback on psychotherapy outcomes? The studies in this special section explore these questions using a wide variety of methods and significantly expand the reach of studies on feedback. Together, these studies represent a snapshot of a maturing field of study: Initial discoveries are developed into more robust theories and applied in a wider range of contexts, while the limits of that theory are tested. They also signal directions for future clinical and research work that may improve patient care in psychosocial interventions into the future.

Multilevel versus single-level factor analysis: Differentiating within-person and between-person variability using the CCAPS-34.

Objective: Although most self-report measures of distress are intended to assess time-varying constructs, they are usually developed using between-person data. They are therefore vulnerable to misspecification due to measurement nonequivalence at the between-person and within-person levels. In recent years, multiple studies have found that self-report distress may not be the same when considered over time versus between people: what changes over time may not be the same as what makes individuals different from one another. Method: In this study, we present a multilevel factor analysis (MFA) of a widely used multidimensional self-report measure of psychological symptoms, the Counseling Center Assessment of Psychological Symptoms-34 (CCAPS-34), in two samples (Ns: 1,223 and 757) of individuals with 10 or more observations each. We compare the results to traditional factor analysis. Results: Single-level factor analyses converged with the established seven-factor structure, regardless of sample or data subset. The MFA largely, but not entirely, recovered the existing factor structure of the CCAPS-34 at the within-person level in both samples, but not at the between-person level. The between-person factor structure was simpler than the within-person factor structure, particularly in the nonclinical sample in which only two factors were sufficient. Conclusions: The factors of this instrument that change over time appear to be narrow, while differences between people are broader. This argues against using general distress measures when assessing treatment outcomes. MFA is a promising method for measure development, even in data with relatively few observations per person. This method may clarify how self-report psychopathology manifests over time. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Effectiveness of routine psychotherapy: Method matters.

OBJECTIVE: Though many studies have shown that psychotherapy can be effective, psychotherapy available in routine practice may not be adequate. Several methods have been proposed to evaluate routine psychological treatments. The aim of this paper is to demonstrate the combined utility of complementary methods, change-based benchmarking, and end-state normative comparisons, across a range of self-reported psychological symptoms. METHOD: Benchmarks derived from randomized controlled trials (RCTs) and normative comparisons were used to evaluate the effectiveness of psychotherapy in a large (N = 9895) sample of clients in university counseling centers (UCCs). RESULTS: Overall, routine psychotherapy was associated with significant improvement across all symptoms examined. For clients whose initial severity was similar to RCT participants, the observed pre-post effect sizes were equivalent to those in RCTs. However, treatment tended to lead to normative end-state functioning only for those clients who were moderately, but not severely, distressed at the start of psychotherapy. CONCLUSIONS: This suggests that although psychotherapy is associated with an effective magnitude of symptom improvement in routine practice, additional services for highly distressed individuals may be necessary. The methods described here comprise a comprehensive analysis of the quality of routine care, and we recommend using both methods in concert. Clinical or methodological significance of this article: This study examines the effectiveness of routine psychotherapy provided in a large network of counseling centers. By comparing multiple established methods to define outcomes in this sample we provide a detailed understanding of typical outcomes. The findings show that, across several different problem areas, routine psychotherapy provided substantial benefit, particularly to clients in the most distress. However, there is room to improve, especially by increasing the number of clients who return to normal functioning by the end of treatment. Using distinct methods provides complementary answers to the question: How effective is routine psychotherapy?

Assessing and investigating clinicians' research interests: Lessons on expanding practices and data collection in a large practice research network.

Conducted in naturalistic settings, practice-oriented research (POR) is aimed at building stronger connections between the science and practice of psychotherapy. Promoting the principles of POR, this article has 2 aims: (a) presenting the results of a survey assessing the interests of members of a large practice research network in topics that could guide future research conducted as part of clinical routine, and (b) describing difficulties in implementing a study in line with such interests. Despite the significant interest in and perceived clinical significance of two relationship constructs (alliance and countertransference), there were unique obstacles faced in their empirical investigation within an already operationally functional practice research network. Challenges in this process included resource-related difficulties (such as changes in staff and the time required to set up the study and administer the measures), logistics-related issues, and effectively incorporating the assessment procedure into an existing clinical system. The article also describes strategies to address these obstacles, with differing degrees of success, including the role of a "local champion" at each site, the importance of a personal/professional relationship between the researcher and participating centers, as well as the pragmatic assistance to sites during the preparation, coordination, and implementation process (e.g., providing templates and feedback on institutional review board applications, and technological assistance on how to incorporate the measures into existing center software). The article concludes with general recommendations and future directions for POR. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Physical, functional, and psychosocial recovery from burn injury are related and their relationship changes over time: A Burn Model System study.

OBJECTIVE: While burn injuries can have dramatic effect on patients' physical health, they can also lead to intense psychological distress, loss of important social and role functioning, and alterations in outward appearance. We aimed to identify potential leading indicators of recovery in the post-discharge period following acute burn injury and hospitalization. METHOD: Using data derived from the Burn Model System National Database, we identified five outcomes of interest measured at four time points (Pre-burn/Discharge, 6 months, 12 months, and 24 months post-discharge), including mental health, physical functioning, community integration, life satisfaction, and satisfaction with appearance. We applied cross-lagged panel analysis to the sample of 1052 injured patients admitted to burn intensive care units. RESULTS: Overall, there was little mean change in the five measures after the 6-month assessment. A time-varying panel model was superior to a time-invariant model, showing that the process of recovery itself may change over the course of two years post-burn. Physical functioning is an important predictor throughout this period, while satisfaction with appearance is quite predictive of other factors at discharge, but somewhat less predictive later. Overall mental health functioning is less predictive at discharge but at later intervals was a meaningful leading predictor of the other outcomes. CONCLUSIONS: Recovery from burn injury is complex and the most important facets of recovery change over time. Future research should focus on developing treatments to help patients adjust to post-burn appearance in the early post-discharge period, and mental health interventions may be effective as patients progress in recovery.

Interpersonal pathoplasticity and trajectories of change in routine adolescent and young adult residential substance abuse treatment.

BACKGROUND: Partnerships between mental health care stakeholders provide a context for generalizable clinical research with implications for quality improvement. In the context of a partnership between an adolescent residential substance abuse disorder (SUD) treatment center and clinical researchers, stakeholders identified knowledge gaps (internal and the field broadly) with regard to patient interpersonal factors that influence working alliance and acute SUD residential treatment outcome trajectories. OBJECTIVE: To (a) examine interpersonal pathoplasticity and identify interpersonal subtypes in a naturalistic sample of adolescent and young-adult patients presenting for routine residential SUD treatment and (b) investigate the association between identified interpersonal subtypes and working alliance and acute treatment outcome trajectories. METHOD: N = 100 patients (Mage = 17.39 years, 68% male, 84% White) completed self-reports of symptom and functioning outcomes, interpersonal problems, and the working alliance on multiple occasions between admission and discharge. Multiple methods were used to identify interpersonal subtypes and test pathoplasticity. Interpersonal subtype was entered as a predictor in respective multilevel models of working alliance and symptom outcome. RESULTS: Interpersonal subtypes of vindictive and exploitable patients demonstrated pathoplasticity. Subtype did not predict working alliance trajectories; however, a significant interaction between interpersonal subtype and a quadratic effect for time demonstrated that exploitable patients with longer than average treatment lengths experienced attenuated symptom change over the course of treatment whereas vindictive patients appeared to demonstrate steady progress. CONCLUSIONS: Interpersonal assessments should be integrated into residential SUD treatment to identify patients with an exploitable interpersonal style who might require additional attention or alternative interventions. (PsycINFO Database Record

Therapist effects and the impacts of therapy nonattendance.

Although dropout from psychotherapy has received substantial attention, the impacts of nonattendance on client outcome across a course of psychotherapy have not been well researched. All in-person psychotherapy treatments require clients to actually attend sessions to generate positive symptomatic results, and missed sessions have at least a time and financial cost. Furthermore, it is plausible that therapist differences exist for client attendance rates. The present study examined impacts of nonattendance, particularly early in a course of treatment, comparing the effects of canceled and no-showed appointments on overall symptom reduction and rate of change while accounting for therapist effects. Using multilevel hierarchical regression, the impact of nonattendance on symptom reduction and rate of change was modeled on 5,253 clients (67.2% female, 72.3% white) across 83 therapists gathered from a practice research network. Results suggested that no-shows, but not cancellations, had negative impacts on the magnitude and rate of symptom change, with larger effects when occurring before the third session. Therapist effects on attendance also were identified; therapists varied greatly on nonattendance percentages of their clients after the third attended session. (PsycINFO Database Record

Psychotherapists' outcomes with White and racial/ethnic minority clients: First, the good news.

The purposes of this study were to (a) investigate whether psychotherapists differ in their effectiveness with clients, (b) determine whether disparities exist within therapists' caseloads in their outcomes with White and racial and ethnic minority (REM) clients, (c) explore therapist factors that might contribute to observed therapist effects, and (d) identify whether treatment outcomes varied for REM and White clients. A sample of 3,825 clients seen by 251 therapists at 45 college counseling centers completed the Counseling Center Assessment of Psychological Symptoms at the beginning and end of individual psychotherapy. Therapists differed in their effectiveness at reducing general distress across clients, and evidence was found for disparities within therapists' caseloads in their effectiveness with REM and White clients. Effect sizes were small. Disparities within therapists' caseloads were not a function of any therapist variable that was studied. Therapy outcomes were similar for White and REM clients. Therapist multicultural competence can, and should, be considered in terms of measurable outcomes across client racial/ethnic groups. It is possible to identify multiculturally expert therapists who evidence competence with both REM and White clients and who might serve as models from whom the field could learn.

Enhancing feedback for clinical use: Creating and evaluating profiles of clients seeking counseling.

The current study explored the reliability and clinical utility of a method designed to identify latent classes of students seeking counseling, based on 8 symptom domains and their interactions. Participants were over 50,000 college students in counseling, assessed with the CCAPS-62 and -34 as part of routine clinical care. Latent profile analysis was used to group an exploratory and confirmatory sample of students by reported symptoms across the 8 CCAPS subscales. Profiles were evaluated for reliability and clinical utility, in particular for risk assessment and the prediction of treatment duration and success. Nine reliably stable latent profiles, or groups of profiles, emerged from analysis. Profiles differed significantly in reported symptoms, demographic makeup, psychosocial history, and diagnoses. Additionally, profiles appeared to capture meaningful differences between clients that had implications for relative risk of suicide, self-harm, and violence toward others as well as significant differences in the number of sessions in treatment and the effect size of treatment. Latent profiles of patients appear to capture meaningful, stable differences that could be implemented in an automated system of evaluation and feedback, and that might be useful to clinicians, administrators, and researchers.

The Counseling Center Assessment of Psychological Symptoms (CCAPS): Merging clinical practice, training, and research.

The goal of this article is to present information about a standardized multidimensional measure of psychological symptoms, the Counseling Center Assessment of Psychological Symptoms (CCAPS; Locke et al., 2011; Locke, McAleavey, et al., 2012; McAleavey, Nordberg, Hayes, et al., 2012), developed to assess difficulties specific to college students' mental health. We provide (a) a brief review and summary of the psychometric and research support for the CCAPS; (b) examples of the use of the CCAPS for various purposes, including clinical, training, policy, and counseling center advocacy; and (c) implications of the integration of routine outcome monitoring and feedback for the future of training, research, and clinical practice. In particular, the article emphasizes how the assimilation of and symbiotic relationship between research and practice can address the scientist-practitioner gap.

Attachment history as a moderator of the alliance outcome relationship in adolescents.

The role of the alliance in predicting treatment outcome is robust and long established. However, much less attention has been paid to mechanisms of change, including moderators, particularly for youth. This study examined the moderating role of pretreatment adolescent-caregiver attachment and its impact on the working alliance-treatment outcome relationship. One hundred adolescents and young adults with primary substance dependence disorders were treated at a residential facility, with a cognitive-behavioral emphasis. The working alliance and clinical symptoms were measured at regular intervals throughout treatment. A moderator hypothesis was tested using a path analytic approach. Findings suggested that attachment to the primary caregiver moderated the impact of the working alliance on treatment outcome, such that for youth with the poorest attachment history, working alliance had a stronger relationship with outcome. Conversely, for those with the strongest attachment histories, alliance was not a significant predictor of symptom reduction. This finding may help elucidate alliance-related mechanisms of change, lending support for theories of corrective emotional experience as one function of the working alliance in youth psychotherapy.

Building a practice research network: obstacles faced and lessons learned at the Center for Collegiate Mental Health.

OBJECTIVE: The Center for Collegiate Mental Health (CCMH) was created through a grass-roots initiative among university and college counseling centers to standardize assessment procedures, conduct empirical studies, and advocate clinical services. METHOD: At present, CCMH has over 240 college counseling center members and oversees a research infrastructure based on these centers' routine services, describing approximately 90,000 individual clients annually. These data are used to provide clinical tools, which can be useful for ongoing clinical services as well as program evaluation, quality assurance, and advocacy on behalf of the counseling centers and clients. RESULTS: There have been substantial obstacles to overcome, and there remain numerous challenges in day-to-day operations. This article provides a brief overview of the challenges and current solutions. CONCLUSIONS: Large-scale collaborations between researchers and practitioners are possible, and some recommendations can be made based on the experience of CCMH.